Skip to content

fix(llm): preserve OpenAI tool call extra content - #78

Open
hongxing1227 wants to merge 1 commit into
openJiuwen-ai:developfrom
hongxing1227:agent/preserve-openai-tool-call-extra-content
Open

fix(llm): preserve OpenAI tool call extra content#78
hongxing1227 wants to merge 1 commit into
openJiuwen-ai:developfrom
hongxing1227:agent/preserve-openai-tool-call-extra-content

Conversation

@hongxing1227

@hongxing1227 hongxing1227 commented Jul 21, 2026 •
edited by openjiuwen-ci-bot Bot
Loading

Copy link
Copy Markdown

Paired: GitHub #78 ↔ GitCode !2018

fix(llm): preserve OpenAI-compatible tool call extra content

Problem

Some OpenAI-compatible providers attach protocol metadata to an individual
tool call under tool_calls[].extra_content. Gemini 3 uses
extra_content.google.thought_signature and requires that signature to be
returned unchanged when the conversation continues after a function call.

The OpenAI SDK retains this field, but OpenAIModelClient converted each SDK
tool call into agent-core's ToolCall using only id, type, name,
arguments, and index. The metadata was therefore discarded before
BaseModelClient rebuilt the assistant message for the next request. A Gemini
3 function-calling loop then failed on the continuation request with a 400
error for the missing thought signature.

Root cause and fix

This change treats extra_content as optional, opaque protocol metadata rather
than adding Gemini-specific detection or manufacturing a signature:

  • add optional extra_content to the internal ToolCall schema;
  • extract it from OpenAI SDK objects, including Pydantic model_extra;
  • preserve it through nested message conversion and streaming chunk merges;
  • serialize it back into conversation history only when present.

Standard OpenAI tool calls remain unchanged because no extra_content key is
emitted when the field is absent.

Reproduction

On an unmodified clean checkout of upstream/develop, the new round-trip
regression test fails because extra_content is absent after:

  1. parsing an OpenAI-compatible tool call;
  2. storing it as an agent-core AssistantMessage;
  3. serializing that message for the next model request.

With this change, the metadata survives the complete round trip.

Validation

  • pytest tests/unit_tests/core/foundation/llm -q: 301 passed, 3 skipped
  • Added coverage for non-streaming responses, OpenAI SDK model objects,
    streaming chunk merges, parallel tool calls, nested message conversion, and
    standard tool calls without extension metadata.
  • Real two-request tool loop against gemini-3.1-flash-lite through Google's
    OpenAI-compatible endpoint: first request returned a tool call and signature;
    the parsed and serialized metadata matched; the continuation request with the
    tool result completed successfully.
  • ruff check, codespell, and git diff --check pass for the change; the new
    test also passes isolated mypy checking.

Reference: https://ai.google.dev/gemini-api/docs/generate-content/thought-signatures

@hongxing1227
hongxing1227 marked this pull request as ready for review July 21, 2026 15:56
@openjiuwen-collaboration-bot

openjiuwen-collaboration-bot Bot commented Jul 21, 2026 •
edited
Loading

Copy link
Copy Markdown

head_sha: c516d08068fe2d6b2280a15256bcaec7d975630c

变更摘要

此 PR 主要为 OpenAI 兼容的工具调用增加对 extra_content（不透明提供方元数据）的完整生命周期支持，确保在解析响应、流式合并、序列化以及对话历史往返过程中，附加在 tool call 上的提供方元数据不会丢失。

主要改动

  • ToolCall 模型新增 extra_content 字段：在 openjiuwen/core/foundation/llm/schema/tool_call.py 中为 ToolCall 类增加 extra_content: Optional[dict[str, Any]] = None 字段，用于承载不透明的提供方元数据（如 Google 的 thought_signature）。
  • OpenAIModelClient 新增元数据提取方法：新增静态方法 _extract_tool_call_extra_content，从 OpenAI 风格的 tool call 对象中提取 extra_content，兼容直接属性、model_extra 字典嵌套以及 Pydantic 模型等三种来源形式。
  • 响应解析和流式解析接入 extra_content：在 _parse_response 和流式 chunk 解析两处创建 ToolCall 时，调用 _extract_tool_call_extra_content 提取并传入 extra_content。
  • BaseModelClient 消息序列化保留 extra_content：在 _convert_messages_to_dict 中将 extra_content 写入序列化后的工具调用字典，确保向 API 回传对话历史时元数据不丢失。
  • AssistantMessage 与 AssistantMessageChunk 完整支持 extra_content：在消息初始化验证（model_validate）、模型导出（model_dump）、分块构造、合并逻辑以及追加逻辑中，均对 extra_content 进行正确的读取、保留与传递。

@openjiuwen-collaboration-bot

openjiuwen-collaboration-bot Bot commented Jul 21, 2026 •
edited
Loading

Copy link
Copy Markdown

head_sha: c516d08068fe2d6b2280a15256bcaec7d975630c

代码审查

审查完成 — 汇总

我已完成对全部 6 个变更文件的逐一审查。以下是审查结论：

审查覆盖确认

文件 审查结果
openjiuwen/core/foundation/llm/model_clients/base_model_client.py 无问题
openjiuwen/core/foundation/llm/model_clients/openai_model_client.py 发现 1 个 P2 问题
openjiuwen/core/foundation/llm/schema/message.py 无问题
openjiuwen/core/foundation/llm/schema/message_chunk.py 发现 1 个 P3 问题
openjiuwen/core/foundation/llm/schema/tool_call.py 无问题
tests/unit_tests/core/foundation/llm/test_openai_tool_call_extra_content.py 无问题

发现问题按严重度统计

  • P0: 0
  • P1: 0
  • P2: 1 — _extract_tool_call_extra_content 缺少对 model_dump() 的异常保护
  • P3: 1 — 流式合并 extra_content 时 or 运算符对空 dict 的处理缺陷

整体风险评估

此 PR 的变更范围紧凑且目标明确：在 ToolCall 模型中新增 extra_content 字段以保留 OpenAI 兼容提供商的附加工具调用元数据，并在序列化/反序列化、流式合并的全链路中正确传递该字段。核心逻辑是正确的，测试覆盖了关键场景（直接属性、model_extra、OpenAI SDK 类型、流式合并、并行工具调用、无 extra_content 的标准路径）。

唯一的可靠性隐患是 _extract_tool_call_extra_content 中 model_dump() 调用无异常保护（P2），建议加 try/except 以优雅降级。message_chunk.py 中 or 的语义瑕疵（P3）在实际场景中触发概率极低，影响可控。整体风险为低。

⚠️ 已识别出整体风险，但无法提取行内评论，请参考整体评估。

@openjiuwen-collaboration-bot

openjiuwen-collaboration-bot Bot commented Jul 21, 2026

Copy link
Copy Markdown

head_sha: c516d08068fe2d6b2280a15256bcaec7d975630c

The pipeline(pipeline number:2018) is running. Please wait a moment...

@openjiuwen-collaboration-bot

openjiuwen-collaboration-bot Bot commented Jul 21, 2026

Copy link
Copy Markdown

head_sha: c516d08068fe2d6b2280a15256bcaec7d975630c

任务名称 结果 日志操作
静态检查 ✅SUCCESS 点此跳转
防投毒检查 ✅SUCCESS 点此跳转
开源合规检查 ✅SUCCESS 点此跳转
UT测试 ✅SUCCESS 点此跳转
ST测试 N/A N/A
build 编译包 N/A N/A
ruff codecheck ✅SUCCESS N/A

@hongxing1227

hongxing1227 commented Jul 27, 2026 •
edited
Loading

Copy link
Copy Markdown
Author

已根据两份检视报告完成处理，更新提交：dce9e0e1。

已修改

  • 并行流式 tool call 合并：不再只与最后一个调用合并。有非空 id 时按 id 匹配；后续片段缺少 id 时按 index 匹配；两者都缺失时才保留原有的最后调用 fallback。修复了并行调用参数片段及 extra_content 可能合并到错误 tool call 的问题。
  • 空字典语义：将 incoming.extra_content or existing.extra_content 改为显式 is not None 判断，区分“字段缺失（None）”和“显式空对象（{}）”。
  • metadata 合并规则：extra_content 作为 opaque provider metadata，保留最新的完整非 None 值，不解释或深合并内部字段。
  • 回归测试：新增 Pydantic metadata 正常 model_dump()、显式空字典、并行流式按 index 合并且签名保持原位置三类测试。

未修改及理由

  • 未给 model_dump() 增加吞异常的 try/except：OpenAI SDK 的实际 extra_content 是 dict，正常路径不会调用 model_dump()；合法 Pydantic model 的转换已有测试。捕获后返回 None 会静默丢失必需的 thought signature，将明确的本地错误推迟为后续 provider 400。
  • 未声明依赖 Pydantic v2：Pydantic v1 的 extra field 可通过直接属性读取；model_extra 只是存在时使用的兼容 fallback。
  • 未做深合并：框架不知道 provider metadata 的内部 schema，深合并可能构造 provider 从未返回过的数据。
  • 未使用 copy.deepcopy()：当前链路只读，没有实际引用污染；深拷贝会增加额外成本及自定义对象失败路径。
  • 未修改 mirror PR body：原 GitHub PR body 已包含完整技术背景；mirror 描述同步不属于本次代码提交。
  • 未补纯类型守卫测试（model_extra=None、整数/字符串/列表）：现有 isinstance(..., dict) 分支已直接覆盖，优先补充了会影响协议正确性的组合场景。

验证

  • 针对性测试：40 passed
  • 完整 LLM 单测：304 passed, 3 skipped
  • Ruff：通过
  • Mypy：通过
  • git diff --check：通过

@openjiuwen-collaboration-bot

openjiuwen-collaboration-bot Bot commented Jul 27, 2026

Copy link
Copy Markdown

head_sha: c591021cb3cc986af41d5ad12d4fdfc0fb7a1c33

任务名称 结果 日志操作
静态检查 ✅SUCCESS 点此跳转
防投毒检查 ✅SUCCESS 点此跳转
开源合规检查 ✅SUCCESS 点此跳转
UT测试 ✅SUCCESS 点此跳转
ST测试 N/A N/A
build 编译包 N/A N/A
ruff codecheck ✅SUCCESS N/A

@hongxing1227
hongxing1227 force-pushed the agent/preserve-openai-tool-call-extra-content branch from c591021 to dce9e0e Compare July 28, 2026 12:01
@openjiuwen-collaboration-bot

openjiuwen-collaboration-bot Bot commented Jul 28, 2026

Copy link
Copy Markdown

head_sha: dce9e0e12f04b903e7c59da807a4e2e83f7ab858

任务名称 结果 日志操作
静态检查 ✅SUCCESS 点此跳转
防投毒检查 ✅SUCCESS 点此跳转
开源合规检查 ✅SUCCESS 点此跳转
UT测试 ✅SUCCESS 点此跳转
ST测试 N/A N/A
build 编译包 N/A N/A
ruff codecheck ✅SUCCESS N/A

@openjiuwen-collaboration-bot

openjiuwen-collaboration-bot Bot commented Aug 4, 2026

Copy link
Copy Markdown

head_sha: dce9e0e12f04b903e7c59da807a4e2e83f7ab858

Merge Verification Failed

@xgg1227, this pr is not mergeable and the reasons are below:

Not Enough Labels
  • the pull request needs 1 approved labels, but now gets 0.
  • the pull request needs 2 lgtm labels, but now gets 0.
Label Usage Tips
  • approved: Agree to merge the pull request. This can be accomplished by commenting /approve
  • lgtm: The code has been reviewed. This can be accomplished by commenting /lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

ci-successful

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@hongxing1227